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(57) Abstract: Disclosed is a lossless image streaming system for the transmission of images over a communication network. The 
fmJi, system eliminates the necessity to store a compressed version of the original image, by losslessly streaming ROI data using the 
original stored image. The imaging system of the present invention also avoids the computationally intensive task of compression 
of the full image. When a user (1 10) wishes to interact with a remote image, the imaging server (120) performs a fast preprocessing 
step in near real lime after which it can respond to any ROI requests also in near real time. When a ROI request arrives at the server, 
a sophisticated progressive image encoding algorithm is performed, but not for the lull image. Instead, the encoding algorithm is 
O performed only for the ROL Since the size of the ROI is bounded by the size and resolution of the viewing device at the client and not 
by the size of the image, only a small portion of the full progressive coding computation is performed for a local area of the original 
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BACKGROUND O F THE INVENTION 
FIELD OF THE INVENTION 



This invention relates to systems and methods for transmission of still images 
over relatively low-speed coi33munication chaimels. More specifically the invention 
relates to progressive image streaming over low speed commimication lines, and may . 
be applied to a variety of fields and disciplines, including commercial printing and 
25 medical imaging, among others. 



BRIEF DESCRIPTION OF THE PRIOR ART 

30 In a narrow bandwidth environment, a simple transfer to the client computer of 

any original image stored in the server's storage is obviously time consuming. In many 
cases the user only wishes to view a low resolution of the image and perhaps a few 
more high-resolution details, in these instances it would be inefficient to transfer die 
full image. This problem can be overcome by storing images in some compressed 

35 formats. Examples for such formats are standards such as Progressive JPEG (W. 
Pennebaker and J. Mitchel, "JPEG, still image data compression standard", VNR, 
1993) or the upcoming JPEG2000 (D. Taubman, **High performance scalable image 
compression with EBCOT', preprint, 1999). These formats allow progressive 
transmission of an image such fliat the quality of die image displayed at the client 

40 computer improves during the transmission. 
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5 In some application such as medical imaging, it is also necessary that 

whenever the user at the client computer is viewing a portion of the highest resolution 
of the image, the jnogressive streaming will temiinate at lossless quality. This means 
that at the end of progressive transtnission the pixels rendered on the screen are exactly 
the pixels of tibe original image. The current known "state-of-the-arf * wavelet 
10 algorithms for progressive lossless stremiing all have a major drawhack: their rate- 
distortion behavior is very inferior to the 'lossy** algorithms. The implications are 
serious: 

1 . Whenever the user is viewing any low resolution of the image (at low 

15 resolutions the term **lossless" is not well defined) more data needs to be sent 

for the same visual quality. 

2. Dxiring the progressive transmission of the highest resolution, before lossless 
quality is achieved, more data needs to be sent for the same visual quality. 

20 

Researchers working in this field are troubled by these phenomena. As F. Sheng, A. 
Bilgin, J. Sementilli and M. W. Marcellin say in [SBSM]: **. . .Improved lossy 
performance when using integer transforms is a pursuit of our on-going work.*' Here is 
an example: 

25 





Rate (bit per pixel) 


Wavelet 


0.1 


0.2 


0.5 


0.7 


1.0 


Floatmg Point 7x9 


24.18 


26.65 


31.64 


34.17 


36.90 


Reversible (4,4) 


23.89 


26.41 


31.14 


33.35 


35.65 



Table 1 — Comparison of the lossy compression performances 
(implemented by the (7,9) Wavelet) to a lossless compression 
Omplemented by a reversible (4,4) Wavelet) of "Barabara'' Image 
30 (PSNR (dB)) ([SBSM]). 
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5 As one can see from Table 1, state of the art progressive lossless coding is 

inferior to lossy coding by more than 1 dB at the high bit-rate. 

Indeed, intuitively, the requirement for lossless progressive image transmission 
should not effect the rendering of lower resolutions or the progressive "^lossy" rendering 
of the highest resolution before lossless quality is obtained. The final lossless quality 
10 should be a layer that in some sense is added to a lossy algorithm with minor (if any) 
effect on its performance. 

The main problem with known lossless wavelet algorithms, such as SPIHT 
[SPl] and CREW [ZASB], is that they use special 'Integer To Mteger*' transforms (see 
**Wavelet transforms that map integers to integers", A. Calderbank, L D'aubechies, 
15 Sweldens, B. L. Yeo, J. Fourier Anal. AppL, 1998), These transforms mimic 
'Mathematically proven" transforms that work well in lossy compression using 
floating-point arithmetic implementations. But because they are constraint to be 
lossless, they do not £q>proximate their floating-point ancestors sufficiently welL 
Although in all previous woifc there have been attempts to correct this approximation in 
20 the progressive coding stage of the algorithm, the bad starting point, an inefficient 
transform, prevented previous authors to obtain decent rate-distortion behavior. 

Our algorithm solves the rate-distortion behavior problem. Using the fact that 
images are two-dimensional signals, we introduce new 2D lossless Wavelet transforms 
that approximate much better their lossy counterparts. As an immediate consequence 
25 our lossless progressive coding algorithm has the same rate-distortion of a lossy 
algorithm during the lossy part of the progressive transmission. 

SUMMARV OV TWR. INVENTION 
The imaging system that is described below is directed to a lossless hnage 
30 streaming system that is different firom traditional compression systems and overcomes 
the above problems. By utilizing a lossless means of progressive transmission means 
the pixels rendered on the screen at the end of transmission are exactly the pixels of the 
original image that was transmitted. The imaging system disclosed herein eliminates 
the necessity to store a compressed version of the original image, by streaming ROI 
35 data using the original stored image. The imaging system of the present invention also 
avoids the computationally intensive task of compression of the full image. Instead, 
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5 once a user wishes to interact with a remote image, the imaging server performs a fast 
preprocessing step in near real time after which it can respond to any ROI requests also 
in near real time. When a ROI request aztives at the server, a sophisticated progressive 
image-encoding algorithm is performed, but not for flie full image. Instead, the 
encoding algorithm is i>erfoimed only for the ROI. Since the size of the ROI is bounded 

10 by the size and resolution of flie viewing device at the cli^t and not by the size of the 
image, only a small portion of the full progressive coding computation is performed for 
a local area oftheorigmal image. This local property is also tme for the client. The 
client computer performs decoding and rendering only for ROI and not the full image. 
This real time streaming or Pixels-On-Demand™ architecture requires different 

1 5 approaches even to old ideas. For example, similarly to some prior art, the present 
imaging system is based on wavelets. But while in other systems wavelet bases are 
selected according to their coding abiUties, the choice of wavelet bases in the present 
imaging system dqpends more on their ability to perform well in the real time 
framework. The system of the present invention supports several modes of progressive 

20 transmission: by resolution, by accuracy and by spatial order. 

BRIEF DES CRIPTIQN OF THE DRAWINOS 

Figure 1 is a system architecture block diagram. 
25 Figure 2 is an imaging system workflow diagram. 

Figure 3 is a flow diagram representing a 'lossless progressive by accuracy*' 
request Hst for a ROL 

Figure 4 is a diagram depicting the client '^progressive by accurac3^* workflow. 

Figure 5 is a diagram d^icting the server wotkflow. 
30 Figure 6 is a diagram describing the server-preprocessing step. 

Figure 7 is a diagram describing the low resolution encoding process. 

Figure 8 is a diagram describing the ROI high resolution processing. 

Figure 9 is a diagram depicting the local forward lossless wavelet transform. 

Figure 1 0 is a diagram depicting the local inverse lossless wavelet transform. 
35 Figure 1 1 is a diagram depicting the progressive rendering steps. 
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5 . . 

5 Figure 12 is a diagram depicting a lossless subband tile whereia the spatial 

gro\q>ing of subband coefficients are at a given resolution and the halfbit matrix is 
associated with the hhj subband. 

Figure 13 a diagram depicting the RGB <-> YUV reversible conversion- 
Figure 14 a diagram depicting a lossless last bit plane data block in which only 
10 hi and hh subband coefficients are scanned. 

Figure 15 is a sample pseudo-code of the encoding algorithm represented by: (a) 
Least significant bit plane scan pseudo-code (b) Halfbit plane scan pseudo-code. 

Figure 16 is a sample pseudo-code of the decoding algorithm represented by: (a) 
Least significant bit plane scan pseudo-code (b) Halfbit plane scan pseudo-code. 
15 Figure 17 is a diagram depicting tiie curve defining the mappmg fiom 

QriginalJhnageJDepth-bit image to ScreenJOepth-bit. 

Figure 18 is a diagram depicting the decomposition of one-dimensional signal 
X to the Low-subband s and the High-subband d and the separable decomposition of 
two-dimensional signal X into 4 matrices (subbands): LLyHL^LH and HH . 
20 Figure 19 is a diagram depicting the first stage of the 2D separable transform 

step in the X-direction. 

Figure 20 is a diagram dqpicting the second stage of the 2D separable transform 
step in the Y-direction. 

Figure 21 is a diagram depicting the application of the fiill 2D Wavelet 
25 transform. 

Figure 22 is a flow diagram representing tiie least significant bit plane scan of 

the encoding algorithm- 
Figure 23 is a flow diagram representing the least significant bit plane scan of 

the decoding algorithm. 
30 Figure 24 is a flow diagram describing a bit plane significance scan of the 

server-encoding algorithm. 

Figure 25 is a flow diagram describing the subband tile extraction of the client 

progressive rendering process. 

Figure 26 is a diagram depicting the preprocessing multi-resolution structure. 

35 
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l>y.TATT.y.n nir. SCRIPTlON OF THR INVTCNTTON 

1. NOTATION AND TERIMONOIiOGY 

The foUowmg notation is used fhroughout this document. 

10 



Tenn 


Definition 


4D 


Four dimensional 


FIR 


Finite Impulse Response 


FWT 


Forward Wavelet Transform 


GUI 


Graphical User Interface 


ID 


Identification tag 


IWT 


Inverse Wavelet Transform 


ROI 


Region Of Interest 


URJL 


TJtdfomi Resource Locator 


LSB 


Least Si^ficant Bit 


RMS 


Root Square Error 


FP 


Floating Point 


PDF 


Probability Distribution Function 



The following terminology and definitions apply throughout this document. 



Term 


Definition 


Rendering 


Procedure to output/display a ROI of an image into a device such 
as a monitor, printer, etc. 


Multiresolution 


A pyramidal structure representing an image at dyadic resolutions, 
beginning with the original image as the highest resolution. 


Subband Transform / 
subband coefficients 


A method of decomposing an image (time domain) to firequeacy 
components (fi-equency domain). A representation of an image as 
a simi of differences between the dyadic resolutions of the image's 
multiresolution. 


Wavelet Transform / 
Wavelet coefficients 


A special case of Subband Transform. 


Progressive 
. Transmission 


Transmitting a given image in successive steps, where each step 
adds more detail to the rendering of the image 


Progressive Rendering 


A sequence of rendering operations, each adding more detail. 


Progressive by accuracy 


Transmit/render strong features first (sharp edges), less significant 
features (texture) last 


Progressive by 
resolution 


Transmit/render low resolution first, high resolution last 


Progressive by spatial 
order 


Transmit/render top of image first, bottom of image last 


Distributed database 


A database ardntecture that can be used in a network environment 
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Subband/Wavelet tile 


A group of subband/wavelet coefficients corresponding to atime- 
firequency localization at some given spatial location and 
resolution/frequency 


Subband/Wavelet data 
block 


An encoded portion of a subband/wavelet tile that corresponds to 
some accuracy layer 



5 

2. OVERVmW OF THE INVENTION 



Referring to Figure 1, a block diagram is provided depicting the various 
components of the imaging system in one embodiment. A client compxxter 1 10 is 
coupled to a server computer 120 through a communication network 130. 

10 In one embodiment, the client computer 1 10 and server computer 120 may 

comprise a PC-type coinputer operating with a Pentium-class microprocessor, or 
equivalent. Each of the computers 110 and 120 may include a cache 111, 121 
respectively as part of its memory. The server may include a suitable storage device 
122, such as a high-capacity disk, CD-ROM, DVI>, or the like. 

15 The client computer 1 10 and server computer 120 may be connected to each 

other, and to other computers, through a communication network 130, which may be 
the Ihtemet, an Intranet (e.g., a local area network), a wide-area network, or the Hke. 
Those having ordinary skill in the art will recognize fliat any of a variety of 
communication networks may be used to itc^lement the present invention. 

20 With reference to Figure 2, the system workflow is described. Using any 

browser type application, the user of the cUent computer 110 connects to the Web 
Server 140 or directly to the Imaging server 120 as described in Figure 1 . He/she then 
selects, using common browser tools an image residing on the Image file storage 122. 
The corresponding URL request is received and processed by the Imaging Server 120. 

25 In case results of previous computations previous conqjutations on the image are not 
present in the Imaging Cache 121, the server performs a fast preprocessing algorithm 
(see §2. 1) in a lossless mode. The result of this conqyutation is inserted into the cache 
121 . Unlike other more '^traditional" applications or methods which perform full 
progressive encoding of the image in an "offline*' type method, the goal of the 

30 preprocessing step is to allow the server, after a relatively fast computational step, to 
serve any ROI specified by the user of the client computer. For example, for a 15M 
grayscale medical image, using the described (software) server, installed on compute 
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5 wilh a Peatimn processor^ fest disk, running Windows NT, the preprocessing step 501 
will typically take 3 seconds. This is by order of magnitude &ster than a "full'* 
conq>ression algorithm such as [S], [SPl], [T|. Serviug ROI is sometimes called 
**pixels-on-demand** which means a progressive transmission of any ROI of the image 
in *YeaI-time", where the quality of the view inq^roves with flie transfer rate imtil 

10 lossless view is received in the cheat side. Once the preprocessing stage is done, the 
server sends to the client a notification that the 'image is ready to be served". The 
server also transmits the basic parameters associated with the image such as 
dimensions, color space, etc. Upon receiving this notification, the client can select any 
ROI of the hnage nsmg standard GUI. The ROI is fcnmulated in step 203 into a request 

15 list that is sent to the server. Each such request corresponds to a data block (§4). The 
order of requests in the list coixesponds to some progressive mode selected in the 
context of the application such as "progressive by accuracy*' rendering of the ROL 
Upon receiving the ROI request list, the server processes the requests according to fhek 
order. For each such request the server checks if the corresponding data block exists in 

20 the Cache 1 2 1 . If not, the server then computes the data block, stores it in the cache and 
immediately sends it to the client Once a data block that was requested arrives at the 
client, it is inserted into the cache 1 1 1 . At various points in time during the transfer 
process, a decision rule invokes a rendering of the ROL Obviously, if some of the data 
blocks required for a high quality rendering of the ROI, were requested from the server, 

25 but have not arrived yet, the rendering of the ROI will be of lower quality. But, due to 
the progressive request order, the rendering quality wiU improve with each received 
data block in an "optimal" way. In case the user changes the ROI, the rendering task at 
the client is canceled and a new request list corresponding to a new ROI, sent from the 
client to the server, will notify the s&rv&c to terminate the previous computation and 

30 transmission task and begin the new one. 

3. NEW REVERSIBlxE WAVELET TRANSFORM 



35 



We will now explain in detail why the rate-distortion behavior of our 
progressive lossless algorithm is better than other known algorithms. Lossless wavelet 
transform, must be integer-to-integer transform, such that round-off errors are avoided. 
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5 In order to demonstrate the difference between lossy and lossless transforms, let us look 
at the simplest wavelet, the Haar wavelet. Let x{k) be the k -th component of the one- 
dimensional discrete signal x . The first forward Haar transform step, in its accurate 
"mathematical*' form, is defined by: 



10 



15 



(3.1) 



where j is a low-resolution version of x , and d is the "diffCTence" between s and x . 
In the case of lossless transform^ applying the above transform results in round-off 
error. One possibility is to apply the transform step suggested by [CSDSI]: 



d (n) = x{2n + l)-x (2n) . 



(3.2) 



The symbol j^o J is the floor fimction meaning "greatest integer less than or equal to o **, 
20 e.g. 

[0.5j==0,L-1.5j = -l, L2j = 2,L~lJ = -l. 

The one-dimensional transform step is generalized to 2D separable transform 
step, by applying the ID transform step twice, first in the X-direction and than (on the 
25 first stage output) in the Y-direction as described in Figure 1 8, 19 and 20 (see also |M| 
7.7). The fuU 2D Wavelet transform is applied by using the 2D Wavelet transform step 
iteratively in the classic Mallat decomposition of the image (Figure 21) {\M\ 7.7). 



30 



In (3,2) two properties are kept: 

1. Reversibility, i.e., one can restore x(2n) and x(2n + 1) , by knowing s(n) and 
d (n) , as follows: 
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10 



x(2n) = s(n)- 



(3.3) 



X (2n + 1) = dr (n) + ;c (2ii) . 



2. De-correlatioii, i.e-, s(^n) and d(^n) contaios the xniiiiinal number of bits 

required in order to follow property 1 . For example^ if the transfbim would have 
been defined by: 



(s{n)^x (2n +1) + jc(2w) , 
d(n) = x(2n-hl)^x{2n). 



(3.4) 



then the least significant bit of s (k) and d (w) would have been the same and 
saved twice. In other words, there is a correlation betwem ,s (») and d (n) in 

(3.4). From the view point of coding this shoidd be avoided since there is a 
1 5 redundancy of transmitting this bit 

On the other hand, the scaling property, that is a very intportant one, is not kept 
in (3.2). Observe that the value of s(n) computed by (3.2), is smaller than its *Veal 

mathCToatical value" as computed in (3.1), by lactor of ^/2 . Since s(n) should be 

rounded to an mteger number, the fact that s(n) is smaller than what it should be, 

20 increases the round-oflf error. In low resolutions, the error is accumulated through the 
wavelet steps. 

If we take the error as a model of *white noise" added to the i—th resolution in 
a multi-resolution representation of the irnage, i.e. in Figure 21, it can be proved 
that the variance of this noise exponentially increases as a fimction of i. This 
25 "contamination*' to the multi-resolution image damages the coding efficiency at low 
bit-rates. Let us describe this in detail for the case of the Haar wavelet We have two 
assumptions in our analysis: 

1. The parity (least significant bit) of an arbitrary coefficient c, in any of the 
wavelet steps is a xmiformly distributed random variable, i.e. 

30 Pr(csO mod2)=Pr(csl mod2)=i. 
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5 2. This parity is indqjendentofother coefficient's parity (Id 

distributed). 

Our referenced computation, i.e* the accurate computation, is the Haar transform 
step defined in (3.1). We concentrate on the LL-subband coefQcients, because the low- 
10 resolution subbands are computed from them. LL-subband coef&cients are the result of 
a 2D-transfomi step (Figure 18) 

^^(accuratc) / ^ :y(2m +l,2n+l)-f jc(2m,2/i+l)+x(2m •H,27?)+jc(2m,2yi) 

where m and n are the indices of the row and column of the coefficient respectively. 
15 As described in Figure 18, 19 and 20, according to the step deJOuied in (3.2), we 

first ^ply the X-direction step 

for each input row x { Jfc, •) , 

Under assumption 1 mentioned above, we can write s (Jt, n) as 



20 



s{k,n) = -i-^ 1 ^ — i+ e, (3.5) 

25 where e is a random variable with a probability distribution fimction (P.D.F.) 
defined by 

p(-0.5) = Pr(e = -0.5) = i 

^ (3.6) 

p(0)=Pr(e = 0) = i 
Therefore, 

30 E(e)=-i Var(e) = ^, (3.7) 
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We th©a apply the Y-direction transfbim by 



s{2m'^lyn) + s(2m,n) \ s{2m + Un) + s{2m,n) 

2 r 2 
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As in (3.5) we can represent s(2m + l,n) and s{2m,n) by: 

/-^ . 1 ^ Jf(2i»+l,2«+l)+j:(2j»+l,2«) 

At 

f_ . x(2m,2n+l)+ x(2m,2n) 

Now we can ymte: 

x(2m + 1, 2w + 1) + a; {2m + 1, 2n) xClm, 2fi + 1) + ;c (2m, 2n) 



t1 ON 



(3.9) 
(3.10) 



_ Jc(2/H4-l,2CT+l)+jc(2m+l,2n)-i-.x(2m,2«-H)-4-.x(2m,2n) ^ e, ^ ^ ^. 



(3.11) 



2 2 



X (2?w +1, 2» +1) + jc (2h» -H, 2n) 3c (2i?t. 2n + 1) + x (2to, 2w) 

4 



where are independent (assuxnption 2 above) random variables with 

20 expectation —.Variance — , and e = -^+-^+e' . 



4 
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2 2 



Therefoie, 
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E(e) = iE(ei)H-lE(^)+E(e-)=-~. 

Var(e) =Var[ |=lvar(ei)+ivar(e,)+ Var(e') = 



(3.12) 



l.J- l.J_ J_ = J_ 
4\6^A\6^16 32" 



Thus, 



U^^-^ {rn,n)Jl ±^^e . (3.13) 

e represents the approximation error of the LL-subband coefficients, results from one 
1 0 2D transform step. The error relates to &e accurate floating-point computation. 

This was a description of a single 2D-traQsform step assmmng that the input 
coefBcients are without any error. Now we wish to evaluate the error accumulated after 
several steps. 

15 At an arbitrary step /^O, we can assume that an input coe£5cient can be 

written as: 

where x}'**'*'^^ (*^>0 is the accurate value achieved by floating-point computation for all 
the previous steps» i.e., a step defined by 

20 2 ' (3.14) 

[d (n) ^x(2n+ l)-x(2n), 

instead of the integer-to-integesr computation in (3.2). Observe that if jc}"*^^ (kj) is 
the i -Ox resolution image coefficient, using (3 . 14) as the ID Wavelet step, then 

^^.)^j^^i^ JiiT^\k,i) .^^^ ^^^^^ 

25 

where l^:^'^{k,l) is the normaUzed (Ij — norm ) LL-subband coefficient resulting 
from the / -th 2D transform step using (3.1) as the ID Wavelet step (see Figure ). e,. is 
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5 the difference between x, (k,l) and x^'^''^ {k,l) (I.e.. the approximation error of the 
integer computation made until now). E.g. c^, = 0 ( jt„ {k,l) is an original image pixel), 

while e, is a random nmnb^witiiracpectation and variance — (see (3 12)) 

2 32 

0, 
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Using (3.1 1), we get; 
///'=»«)(;«,«) 

4 

_ 4'"H2m+l.2n+l)+x^)(2m+l,2n)+xj^H2m,2n + l)+x^^2m,2n) e]+ef+ ej + ^ 

4 + 4 

_ JC^'">(2w+1.2i»+l)+x/"°) (2m+l,2n)+x/->(2w.2»+l)+;>:/">(2m,2i») 



where e,^, is dejQned by 
15 e , - 

T 

4 

and corresponds to ZX^ subband. 
Consequently 

. E(e,,,) = E(e,) + E(e), 

Observe that 

20 E(«)=-i.Var(e) = A. 

As a result, we can write recursive formulas for the error expectation and variance after 
i steps. 
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'B(e„) = 0, 

E(e,,0 = E(e,)-i 

Var(«i,) = 0. 

„ , . Var(e,) 3 



(3.16) 



10 



Hie explicit solutions to these fi>nimlas axe 



Byr^lacing jcj"^''>(w,») with 



we get 



15 



20 



Thus, the approximation to //C*^**"^) (m,n) is 

2M^^lCOSl) ^^^^^ ^ ^^(accunte) ^^^^^ ^ 

The approximation error expectation is 

E(2'«,) = 2'E(e,) == 2' (-1^ = -i2'-\ 
25 The approximation error variance and standard deviation are 



V»r(2'.,)-4'V»(^)-4'(i-^) = ifl4 = 2 
Std(2'e,) = ^Var(2'e,)»-C. 



Hence 



(3.17) 



(3.18) 



30 
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Let us now evaluate the approximation error of the 3 other subbands: 



2' 2 { 2 J 2' ' 



where 



10 



• e* l^k^4 are identical to the random variable whose expectation and 
variance are given in (3.17). 



• e' and e" are identical to the random variable whose expectation and variance 
are given in (3.7). 

15 Thus, 

B(^) = 0. 



^' ^ 4{S 2-4'*' 8 2-4'*" j 16 
Ifl 1 1 1 ^ J_ 
4U 2.4'« 8 2-4'«j'^16 



4 2.4*^' ■ 
The approxinaation to 11^'"^^ {m,n) is 



(3.19) 



20 2'/V°^)(m,«) = /A<™)(«.„) + 2'^. 

The ^proximation error variance and standard deviation are: 



Therefore 



Var(2'^) = 4'Vax(^) = 4' .[i-_l_j = 4- -i«4- 
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5 Std(2^e/^ ) = ^Yar{2'ef^) « ^JaF = 2''\ 

A similar approximation error estimation, it can be done with the HL-subband and the 
HH-subband. 

The approximation error evaluation results are smnmarized in the following 
10 table where the error is the difference between the normalized (in Z^— norm) 
coeflBcients according to [CDSI] reversible transform and the "mathematical" 
transform (defined in (3.1))* 





Expectation 


Variance 


Std 


LLf - error 


-(/+1)2' 


8 


0.707-2' 


LH, - error 


0 


2.4' -1 

a»4 

8 


0.5-2^ 


HLi - error 


-i.2' 
4 


^•^'-^«3.4« 
16 


0.433-2' 


HH, - error 


0 


8 ^ 


0.354-2' 



15 Table 2 - Normalized (in L^-norm) approximation errors of the 

Wavelet coefficients at resolution i>:0 (i = 0 is the highest 
resolution) using the {CDSI) reversible Haar transform. 

Assuming a low-bit rate transmission where only the coefficients whose absolute 
20 value belongs to the range [2*, 2***) are encoded, for every resolution i , where i is 

greater than b (less or more). It muist be noted that the large error implies a significant 
loss of coding efficiency. 

Instead, we propose a new family of reversible transforms. The proposed family of 
integer wavelet transforms has all three properties: 
25 1. Reversibility 
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5 2. De-correlatioii 

3,. ScaliBg - Le. improved approximation of the ^mathematical" transform. 
Our 2D transfoim step is separable also, but the one-dimensional transfoim 
step, which the 2D transfoim is based on, is different for the X-direction (step 1901), 
the Y-direction stq> applied on the low output of the X-direction step (step 2001) and 
10 the Y-direction step applied on the high output of the X-direction step (step 2002) as 
described in Figure 18, 19 and 20. 

The full 2D. Wavelet transform is ^plied by using ttie 2D Wavelet transform 
step iteratively in the classic Mallat decomposition of flie image (Figure 21) ([M] 7.7). 
As mentioned before, the Wavelet coefficients in our proposed transfoim are all scaled, 
15 ie. nonnalized in Z^-norm as the Wavelet coefficients computed in the accurate 
**mathematicar* transfoim. 

In order to achieve the third property (improved approximation of the 
•^mathematical" transform), we define an extra matrix we call the "Half bit-matrix" 
which enables the reversibiKty of the High Y-transform step (step 2002). The elements 
20 that belong to this matrix are bits, such that each bit corresponds to an HH-subband 
coefficient in. the following interpretation. Let us describe this by the following 
example. 
Supposing 

5(«) = 7,rf«(n) = 9 

25 are a coefficient pair results from a reversible de-correlated ID-wavelet step 

>>(n) = ;c(2/»+l)-x(2»), . 

I 2 

Now, (») has to be multiplied by ^ , in order to be scaled. 
The binary form of d^'^ (n) = 9 is 

J<'^(n) = 100Lj. 

30 If we now divide rf^'^ («) by 2 in a floating-point computation we get 
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5 . 

Let XX3 call the bit, which located on the AViite-side of the floating point the **Half 
Bit* \ Observe that the Half Bit of (n) is the LSB of d^'^ (n) . Therefore, an 
equivalent way to do this in an integer computation without loosing the Half-Bit is to 
calculate fiurst the LSB of d^^^ (n) by 

10 HalfBit (n) = d^'^ (72)mod2 9mod2 = 1, 

then to shift-write d^^^ (n) by 

d (w) = rfW » 1 = 1001 » 1 = 100. 
By saving d(n) and HalfBit (ji) we can restore back d^^^{n). 

In the proposed transfonn, this Half-bit is needed in the HH-subband coefficient 
15 computation. Therefore in our wavelet decomposition for every HH-subband 

coefficient (in all scales) there is a corresponding bit, which is the coefficient's Half- 
bit. 

The Half bit matrix is hidden in the HH-subband in the description of Figure 18, 19 and 
20. It is described explicitly in the specification of flie transform as much in the coding 
20 algorithm. 

We now present our integer-to-integer versions of the Haar transform and the 
CDF (1,3) transform for the 2-dimensional case. 



3.1 Reversible Haar and (CDF) (1,3) Transforms 

25 

3.1.1Haar Transform 

With respect to Figure 19: 

3.1.1.1 Step 1901: X-direction 
30 Forward Step 

d{n) = jc(2n + 1) - jc(2«). 
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10 



15 



Inverse Step 



(2„)..(„)-[iMj. 

x{2n+l)^d(n)+x(2n). 



(3.21) 



With respect to Figure 20: 

3.1.1.2 Step 2001: Y-direction - Low Forward Step 

s(n)='x(2n)+x(2n+l), 

\ x(2n+l)-x(2n)\ 
(»)= ^ , (3.22) 

[rf(n) = 2d«(n). 
Remarks: 

1 . .s (/i) is a scaled LL-subband coefficient 

2. s (/i) and d^*^ (n) are de-cotrelated and a reversible couple (can be transformed 
back to x(2n) and x{2n+l)),but d^^^(n) is not scaled (it is half its "real 
value'O. Thus, d^^^ (n) is multiplied by 2. Nevertheless, the LSB of the LH- 
subband coe£Scient d (71) is known to be 0 and not encoded* 



20 



Inverse Step 

x(2n + 1) = (/j) + d {n) + {s (n)mod 2)), 
X (2n) = ^ (n) - X (2n + 1) . 



(3.23) 



With respect to Figure 20: 

25 
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3.1.1.3 Step 2002: Y-direction - High Forward Step 

(n) = x{2n + l)-x{2n), 
HalfBit (n) = (/»))mod2. 



(3,24) 



s{n)^x{2n)'¥d{n). 

Remark: dS^^{n) and s{n) are de-correlated and reversible couples, but dS^^{n) is not 
scaled (It is twice its "real value**). Therefore, d^^^ (n) is divided by 2. By doing that, 

we lose its least significant bit, which cannot be restored. To solve this problem, as 
explained before, we save this bit as the ''Half-Bit", Giving this name to that 

coefGcient means that its weight is — in the "real mathematical scale**, and it is the 

least significant (from the approximation point of view). 
Inverse Step 

x((ln) = s{n)-d{n). 



df^^ (n) = Id (n) + HalfBit {n), 
X (2» + 1) = + ;c(2n). 



(3.25) 



15 



20 



25 
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5 3.1:2 CDF (1^) Transform 
3.1.2.1 Step 1901: X*direction 

With respect to Figure 19: 



Forward Step 



10 



Inverse Step 



x(2/i)-hx(2yi4-l) I 

2 J* 

d^'\n) = x(2n + l)-x(2n), 

d («) = rf« („)-,|^£(^zilz£(^J. 



x{2n) = s{n)- 



x{2n + 1) = x(2n) + </W («). 



With respect to Figure 20: 
3.1.2.2 Step 2001: Y-direction - l.ow Forward Step 

J (n) = x(2«) + jc (2w + 1) , 



15 



£/«(«) = 



x(2ii+l)-;c(2/i) + 



j(/i-l)-j(yt-t-l) 



8 



d(n) = 2J«(«). 



(3.26) 



(3,27) 



(3-28) 



Remai^: See remaiks for (3.22). 
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Inverse Step 



V \ / \ ^(«-l)-^(« + l) I 
(«) = *(«)- -i^ ij. 

x(2n + 1) = (n) + (n) + (n) mod 2)), 



(3.29) 



x(2n) = s(n)-x{2n + i). 



With respect to Figure 20: 
10 3.1.2.3 Step 2002: Y-flirection - High Forward Step 



x(2n)+x(2n+r) 
2 



s(«) = 

rfW („) = ;c(2n +1) -x(2/i). 



4~ 



2 



d{n). 

Hal/Bit (n) = (n) mod 2. 



(3.30) 



15 



Inverse Step 

d^'^ (n) = 2d(n)+HalfBit(n)- 

x(2») = *(n) ^ . 

x{2n +1) = d^'> (») + x{2n). 



4 



(3.31) 



We now compute the £^proxixnation error jirobabilities of our method^ aad 
show that it is significantly smaller. We start with the LL-subband error. Assuming 
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5 is the approximation error of the LL-subband in the / - th resolution (Figure 2 1), the 
LL-subband coe£Scient in the i — tfa resohition can be written as: 



,7(new) 



Jc}"^^ (2m + l,2/i) + jc<""^> (2m,2n) 



10 



15 



20 



2 



^ x}'^>i2m + l,2n)+eU+4'^H2m,2n)+eU „ 
2 "•■^ 

_ ^f"") (2>7,+l,2n + l)-t-x<""> (2m,2»+l)+x<'^)(2m+l,2n)+;c<°°"> (2m.2n) 

2 

2 



(3.32) 



where 



• llj^"^^ (/n, «) is the new transform LL-subband coefficient ( / -th resolution). 

• 1 for 1^ * ^ 4 , are identical random variable representing the error from the 
previous level. 

• e* and e" are random variables with P.D.F. defined in (3.6). 

• x^^^ (m, n) is the i -th resolution image coefGLcient using (3 . 1) as the 1 D 
Wavelet step. 

• 7//**^^ is the normalized (Z^ —norm ) LL-subband coefficient resulting 

from the I -th 2D transform step using (3.1) as the ID Wavelet step (see Figure 
21). 

. e, = <A -^^tl +<i +<i ^g»^.^n^ 



Consequently 



(3.33) 
(3.34) 
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By knowing that e,, = 0 we get 



E(e,) = i-2'. 
Var(e,) = 



i+1 



(3.35) 



Now we can easily evaluate the approximation enor of the 3 other subbands: 



= 2 



{2m + 1.2n +1)+ x}"^) (2m, 2n + 1) 




Ji^-^^ (2ot + 1, 2n) + ;«f"*> (2m, 2 n) 


2 




2 



(3.36) 



(2/n-H.2w+l)+;c<""> (2m,2»H-l))-(x<""> (2m+1.2n)->-x<'^> (2m,2n)) 

2 

+e2 -gS _g* 
2 

= tt<«'^>(.«,«) + ^f«. 



15 



20 



where 



new transform LH-subband coefScient (z -tb resolution). 

• /^(«»»»*») is the noimalized ( - wo/tw ) LH-subband coefficient resulting 

j&om the i -th 2D transfonn step xising (3.1) as the ID Wavelet step (see Figure 
21). 

• e is a random variable with PD.F. defined in (3.6). 

• =fk±£kz5L!Jl£k+e'~e"+2e''' 

2 

• all other symbols are defined Hke in (3.32). 



25 
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15 



Hence 



V ' / A 16 16 o 



16 8 



(3.37) 
(3.38) 



Similar estimation can be done for the HL and ibo HH subbands. 

The raior estimation (for all subbands) are smmnaiized in the following table 
where the error is the difference between the normalized (in —norm) coefficients 

according to our new reversible transform and fhe ^^nathematical" transfcnm (defined 
in (3.1)). 





Expectation 


Variance 


Std 


LL^ - error 


i-2' 


£+1 






2 


8 




LHg - error 




i+3 


li + 3 




2 


8 


V 8 


HL^ - error 


_1^ 








4 






HH^ - error 


1^ 


i+i- 






4 


8 16 





20 



Table 3 - Normalized (in I^-norm} approximation errors of tlie 
Wavelet coefficients at resolution z^O (z = 0 is the highest 
resolution) using the proposed reversible Haar transformu The 
result for the IJL-subband is valid for the proposed reversible (1,3) 
transform also. 



The meaning of this result is that in a low bit-rate, where only large coefficients are 
encoded, this error is negligible. 



25 
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5 4. IMAGmC PROTOCOL AND DISTRIBUTED DATABASE 

Dividing tbe data Into tiles and bit-planes 

For the purpose of efficient rendering the coefficients may be sub-divided into 
tiles. The tiles of this invention differ firom previous art as shown in Figure 12. As in 
10 the lossy algorithm, here also the subband tiles are further decomposed to subbaxid data 
blocks. Bach data block of lossless subband tile (Figiu-e 12) will have a 4D coordinate 

(Lx, t_y, t__resolution,t_bifPlane) 
where 0 ^ t_bitPlane ^ maxBitPlaae(t_resolution). 
Bach data block contains the following data in encoded format: 
15 1. FortJbitPlane ^2: 

eu A Ust of the indices of all subband coefficients whose absolute value is 
in the range [2'-*'tf'^«^i^2'-*"/>^«e), 

b. The sign of all the coefficients of a. 

c. For tJbitPlane >2, an additional precision bit for any coefficient that 
20 belongs to the current bit plane or any higher bit plane. 

2. For tJbitPlane =1, which we call the **least signijOcant bit plane'': 

a* A list of the indices of HL*subband and HH-subband coefficients whose 
absolute value belongs to the set {—1,0,1} • 

b. A "zero flag** for eacdi coefficient of a, which indicates if the coefficient 
25 is equal to zero or not 

c. The sign of all the coefficients of a, whose "zero flag" is false. 

d. The LSB of the HL-subband and HH-subband coefficients that belong to 
higher bit plane. 

Remark: 

30 Since the LH-subband contains only even coefficients, their LSB must be 

zero and is not coded. 

3. For tJbitPlane =0, which we call the '*half bit plane", a matrix of 

^ tilel^gth^ bits associated with HH-subband coefficients as their last **half 
bit** (See (3.24) or (3.30)). 
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5 5. TTHE PROGRESSIVE SUBBAND CODING ALGORITHM 
5.1 The encoding algorithm 



The encoding algorithm of the present invention is perfoimed at flie server 120. In 
flie present imaging system this rather time consuming task is performed locally in near 
1 0 real-time for a ROI, and not on the fiill image. The encoding algoriflmi is described for 
images with a single color component, such as grayscale images, but of course may 
also be applied to images with multiple color components. The straightforward 
generalization for an arbitrary number of components will be explained later. 

The lossless algorithm receive as it^ut tiie following parameters: 

15 



Variable 


Meaning 


coef 


Matrix of subband coefiScients, containing 
^^^leLength^ coefficients 


HalJSit 


Matrix of bits containing ^^^^^^^J 



Table 4 - Lossless Encoding Algorithm Input Parameters 

The coding strategy is similar in some sense to that described in A, Said and W. 
20 Pearlman, "A new, fiast and efficient image codec based on set partitioning**, lEEB 

Trans. Circuits and Systems for video Tech., Vol. 6, No. 3, pp. 243-250, 1996, but the 
preferred embodiment uses no "Zero Tree** data. For all the data.blocks with tj>ifl?lane 
^2, we use the lossy encoding algorithm described in previous art with the parameters: 

• co^ co^ (The lossy parameter coef initialized with the lossless parameter 
25 coef) 

• equalBinSize := Tme 
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5 



Remark: The lossy algorithm encodes all the bit-plane information for 
t_bitPlane^2, 

For t_bitPlane^ 1, i.e. the least significant bit plane (of the lossless algorithm) and the 
1 0 half bit plane, we use a different algorithm described in 5.1 .3. 

5.1*1 Encoding algorithm initialization 

The lossless encoding algorithm initialization is the same as the lossy algorithm 
of § 4.1,1 in the above-cited Ser. No. 09/386^64, which disclosm-e is incorporated 
1 S herein by reference. In order to initialize the encoding algorithm, the following 
procedure is performed: 



1. 




20 



2. 



Compute the maximum bit plane over all such coefficients: 




= max 



25 



3. 



Write the value of meocBitPlane{tile) using one byte as the header of 



the data block: 



{t _x,t _y,t _ resolution, maxBitPlane{t _ resoltOion)) 



4. 



Initialize all the coefficients as members of their corresponding Typel6 



30 



group. 



5. 



initialize a list of significant coefficients to be empty. 
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6. 
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30 

Initialize a coefBcieiit approximatioii matrix coef as zero. 



5.1^ The outer loop 

The outer loop of the encoding algorithm scans the bit planes from 
b = 7naxBitPlane{tile)to 6 = 0. The output of each such bit plane scan is the subband 

10 data block. Since the last stage of the encoding algorithm is arithmetic encoding of 

given symbols, at the beginning of each scan the arithmetic encoding output module is 
redirected to the storage area allocated for the particular data block. Once the bit plauQ 
scan is finished and the data block has been encoded, the output stream is closed and 
the bit plane b is decremented. After the outer loop is finished the following stages are 

15 performed: 

1 . I/east signiBcant bit plane is encoded (t_bitPlane = 1). 

2. Half bit plane is encoded (tJ>itPlane = 0). 

20 The ou^ut of the least significant bit plane scan is the data block (Figure 14): 

(t-.x, t_jr, tjresolution, tJiitPlane^l). 
The half bit plane data block is: 

(*_x> *-y» tjresolutioxi, tJbitPlane=0). 

25 

5.1.3 Bit-plane scan 



For tJbitPlane >2, the firamework of the bit plane scan is described in Figure 
24, while the pseudo code is given in the above-cited Ser. No. 09/386,264, which 
30 disclosure is incorporated herein by reference. The scan, for a given level b (6 2) , 
encodes all of the coefBci^ts' data corresponding to the absolute value interval 



. Remark: The encoder method isLastBitPIaneQ is associated to flie tjbitPlane == 2. 



35 
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5 For the least significant bit plane, a pseudo code is described in Figure 15, while a flow 
chart is described in Figure 22« 

Explanations to the least significant bit encoding algorithm: 

1. The coefficients scanning procedure, i.e. moreCoefQ procedure in Figure 15 or 
•'More coefficients?" in Figure 22 includes aU tiie coefficients belong to the HH and 

10 the HL-subband (Figure 14). The LH-subband is skipped, since the least significant 

bit of each coefficient in it is zero (see remark 2 for (3.22)). 

2. The procedure isCoefReporetedO ('Is coefficient reported?" in the flow chart) 
returns false if the coefficient is one of {-1,0,1} , i.e. in all higher bit plane scans it 
was insignificant, otherwise it returns true. 

15 3. The procedure isCoefBxactZeroQ ("Coefficient is zero?** in the flow chart) returns 
true iff the coefficient is z&ro. 
4. The procediire getCoefSignQ returns the coefficient's sign. 



For the half bit plane, a pseudo code is described in Figure 16. 

20 



5.2 The decoding algorithm 



Obviously, this algorithm is a reversed step of the encoding algorithm of section 
25 5.1, performed in the server 120. The client computer 110 during the progressive 
rendering operation performs the decoding algorithm. Similar to the encoding 
algorithm, the decoding algorithm is described for an image with one component (such 
as a grayscale image), but of course could also be used with an image with more than 
one component. The input parameters to the lossless algorithm are given below: 



30 
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Variable 


Meaning 


coef 


— 1- . 

Empty matrix of subband coefficients to be filled by the decoding algorithm. 


Hal/Bit 


Matrix of bits containing ^^^^^^^j bits. 



Table 5 - Lossless decoding algorithm inpnt parameters 

For all the data blocks with t_bitPlane ^2, a "loss/' decoding algorithm is utilized. The 
input parameters for the lossy algorithm are: 
10 • coef 7= coef 

• equalBinSize True 



Decoding algorithm iDitialization 
15 1. Assign to each coefiScient Coef {x^y) the value zero, 

2. Assign to each bit belongs to the HalfBit matrix the value zero. 

3.1nitialize all the coefficients as members of their corresponding TypelS group. 

4.1hitialize the Hst of significant coefficients to be empty. 

5,If the "firsf* data block 
20 _ ^» ^ _ ^_ resolution, maxBitPlane{t _ resolution)) 

is available at the client, read the first byte, which is flie value of maxBUPlane{tile) . 

5.2.2 The outer loop 

Upon the completion of the outer loop in 5.1.2the following stages are preformed: 
25 1. The decoding algorithm scans the least significant bit plane. The input to this 

stage is encoded data block (t_x, t_y, tjresolution, LeastSiginificant_bitPlane). 
2. The decoding algorithm scans the half bit plane. The input to this stage is 
encoded data block (t_x, t_y, t_resolution, HalfbitPlane). 
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The preferred embodiment follows the lossy prior art of the above-cited Ser. No* 
09/386,264, which disclosure is incorporated herein by reference, for tJbitPlane^2. 
The scan, for a given level b , decodes all of the coefScients' data corresponding to the 
1 0 absolute value interval [^2* , ) . Pseudo codes of the least significant bit plane scan 

and half bit plane scan are described in Figure 16. 
Explanations to the pseudo code: 

1. The decoder's method moreCoefO scans all the coefficients in the HH, HL and LH 
subband. But, since the LH-subband is skipped in the encoding algorithm, we don't 

1 5 call to decodeSymboIO for its coefficients. Instead of this, we update their least 

significant bit as zero. 

2. Recall that LH-subband coefficients that have not been reported until the least 
significant bit-plane must be zero since they are known to be even. 

20 

6. CLIENT WORKFLOW 

With reference to Figure 4, we describe the workflow at the client unit 1 10. Any 
new ROI generated by the user's action such as a zoom-in, a scroll, or a luminance 
tuning invokes in step 401 a call from the GUI interface to the client imaging module 

25 with the new ROI view parameters. The client imaging module then computes in step 
402 which data blocks are required for the rendering of the ROI and checks if these 
data blocks are available in the client cache. If not, their coordinate is added to a 
request Ust ordered according to some progressive mode. The request list is then 
encoded in step 403 and sent to the server. The server responds to the request by 

30 sending back to the client a stream of data blocks, in the order in which they were 
requested. In step 404 the client inserts them to their appropriate location in the 
distributed database. At various points in time step 405, a rendering of the ROI, is 
invoked. Naturally, the rendering operation is progressive and is able to use only the 
currently available data in the client's database. 

35 
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5 6.1 Step 401:Receiving the ROI Parameters 

The imagmg module on the client computer 120 receives from the GUI 
interface module view parameters detailed in Table 5. These parameters are used to 
generate a request list for the ROI. The same parameters are used to reader the ROI. 

10 



Table 5 



Variable 


Meaning 


worldPolygon 


A 2D polygon in the otigiual image coordinate system 


scale 


The view resolution. 0 < scale < 1 implies a low view resolution^ 
scale = 1 original resolution and scale > 1 a higher than original view 
resolution 


deviceDepth 


A number in the set {8, 1 6, 24} representing the depth of the output 
device Cscreen, printer) 


viewQuality 


A quality factor in the range [1,7] where 1 implies very low quality and 
7 implies lossless quality 


luminanceMap 


If active: a curve defining a mapptag of medical images with more than 
8 bits (typically 10,12,16 bits) per grayscale values to an 8 bit screen 


progressiveMo 


One of: Progressive By Accuracy, Progressive By Resolution, 
Progressive by Spatial Order 



The basic parameters of a ROI are worldPolygon and scale which determine 
uniquely the ROI view. If the ROI is to be rendered onto a viewing device with limited 
resolution, then a worldPolygon containing a large portion of the image will be coupled 

20 by a small scale . In the case where the rendering is done by a printer, the ROI could 
be a strip of a proof resolution of the original image that has arrived from the server 
computer 120. This strip is rendered in parallel to the transmission, such that the 
piintingprocess will terminate with the end of transmission. The other view 
parameters determine flie way in which the view will be rendered. The parameters 

25 deviceDepth and viewQuality determinethequaUty of the rendering operation. In 
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5 cases the viewing device is of low resolution or the user sets the quality parameter to a 
lower quality, the transfer size can be reduced significantly. 

The parameter luminanceMap is typically used in medical imaging for 
' grayscale images that are of higher resolution than the viewing device. T3^icallyy . 
10 screens display grayscale images using 8 bits, while medical images sometimes 

represent each pixel using 16 bits. Thus, it is necessary to map the bigger pixel range 
to tiie smaller range of [0, 255] . 

Lastly, the parameter progressiveAdode determines the order in which data 
blocks should be transmitted from the server 120. The '^Progressive By Accurac}^* 
15 mode is the best mode for viewing in low bandwidth environments. 'Trogressive By 
Resolution*' mode is easier to implement since it does not require the more 
sophisticated acciu-acy (bit plane) managemrat and therefore is commonly found in 
other systems. The superiority of flie "progressive by accuracy" mode can be 
mathematically proven by showing the superiority of **non-linear approximation" over 
20 "linear ^proximation" for the class of real-life images. See, e.g., R. A. DeVore, 
**Nonlinear approximation", ActaNumerica, pp. 51-150, 1998. 

The *Trogressive by Spatial Order" mode is designed, for example, for a **print 
on demand" feature where the ROI is actually a low resolution *^roof printf* of a high 
resolution grephic art wotk. In this mode the image data is ordered and received in a 
25 top to bottom order, such that printing can be done in parallel to the transmission. 

However, since lossless compression is mostly required in medical images 
transmission, where typically more than 8 bits images are used, we ampliJ^ our 
discussion on the cmrve (luminanceMap hereinabove) which defines the mapping from 
the original image gray scale range (typically 10,12,16 bits) to an 8-bit screen. Further 
30 more, in medical images viewing, regardless of the original image depths iiaapping is 
required in order to control the brightness and contrast of the image. 



35 



6.1.1 Luminance mapping 

Mapping from original image depth (e.g. 10,12,16 bits ) to screen depth 
(typically 8-bits), is defined by a monotonic function (Figure 17): 
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5 / : 1^0,2"^'-*^-*^ -l] -» j^o,2'~"-'^'* -l]. (6.1) 

The curve influences not only the m^ping» i.e. the drawing to the screen, but also the 
request &om the server. To understand that, let us concentrate in the maximal gradient 
of the curve (Figure 17). In a lossy mode, the request is created such that the image 
10 apprOTcimation in the client side is close enough to the origmal image, i.e., the RMS 
ORoot Mean Square Error) is visually negligible. When a ciurve (mapping ftmction) is 
applied, the RMS can be increased or reduced* The mflYTmal RMS increasing factor 
depends on the maximal gradient of the curve as follows: 

ItMs(/(l),f(f)) 

RMS _mcieasing_f actor = i — ^J^^maxif) , (6.2) 

RMS(l,l) ^ ' 

15 where 

• / is the original image 

• / IS the approximated image 

• y is the mapping function 



Image_size 

20 • max (/ •) is the maximal gradient of the curve. 

We consider the worst case of the RMS increasing factor i,e.: 

RMS _ increasing factor = Maximal _ gradient = max (/ ') 
If the RMS increasing factor is greater than 1, it means that the "new RMS'* may be 
25 greater than we consider as visually negligible error. Thus, the request list should be 
increase (more bit-planes should be requested from the server) in order to improve the 
qiproximation accuracy . Conversely, if the RMS increasing factor is smaller than 1, the 
request listing can be reduced. The exact specificatLon of this is given in the following 
• section. 

30 

6.2 Step 402: Creating the request list 

In step 402 using the ROI view parameters, the client imaging module at the 
client computer 1 10 calculates data blocks request list ordered according to the 
progressiveMode . Given the parameters worldPofygon aad Scale , it may be 
35 detetmined.which subband tiles in the 'frequency domain" participate in the 
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5 reconstruction of ttie ROI in the "time domain*'. These tiles contain all the coefficients 
that are required for an **Inverse Subband/Wavelet Transform" (TWT) step that 
produces the ROI. First, the parameter dyadicResolution^ROl) is computed, which is 
the lowest possible dyadic resolution higher than the resolution of the ROI. Any 
subband tiles of a higher resolution than dyadicResolution (^ROI) do not participate in 

10 the rendering operation. Their associated data blocks are therefore not requested, since 
they are visually insignificant for the rendering of the ROI. If scale > 1 , then the 
highest resolution subband tiles are required. If scale ^ 2^""''""*^^^'^''''" then only the 
lowest resolution tile is required. For any other value of scale we perform the 
mapping described below in Table 6. 

15 

Table 6 



scale 


highestSubbandResolution 


scale ^ '2^—nvn^erOplesolution3 


1 




numberOfResolulioTts — log^ (^ca/e) J 


scale > 1 


numberOfResolufions 



20 Once it has been determined which subband tiles participate in the rendering of 

the ROI, it is necessary to find which of their data blocks are visually significant and in 
what order they should be requested. Using weU known rate/distortion rules firom the 
field of image coding (such as is described in S. MaUat and F. Falzon, 'TJnderstanding 
image transfomi codes", Proc. SPIE Aerospace Conf., 1997), it is not too difficult to 

25 detennine an optimal order in which the data blocks should be ordered by the client . 
imaging module (and thus delivered by the server 120). This optimal order is described 
in steps 301-310 of Figure 3 for the **Progressive By Accurac/' mode. The underlying 
mathematical principal behind this approach is '*Non-Linear Approximation". 

First, the subband coefficients with largest absolute values are requested since 

30 they represent the most visually significant data such as strong edges in the image. 

Notice that high resolution coefficients with large absolute value are requested before 
low resolution coefficimts with smaller absolute value. Witibin each given layer of 
precision (bit plane) the order of request is according to resolution; low resolution 
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5 coefficients are requested first and the coefficients of highestSubbandRespIution are 
requested last 

The main difficulty ofthis step is this: Assmne a subband tile is required for the 
rendering of the ROL This means fliat / _ resolution ^ {fyadicResolution (i?0/) and the 
tile is required in the IWT procedure that reconstructs the ROI, It must be understood 

1 0 which of the data blocks associated with the subband tile represent visually 

insignificant data and thus should not be requested. Siding all of the associated data 
blocks will not affect the quality of the progressive rendering. However, in many cases 
transmitting the **tail" of data blocks associated with high precision is unnecessary 
since it will be visually insignificant. In such a c^e, the user will see Hxat the 

1 5 transmission of the ROI from the server 120 is still in progress, yet the progressive 
rendering of the ROI seems to no longer to change the displayed image. 

Additionally, the influence of the luminance mapping on the accuracy level of 
the r^uested data block as described below. Siq)posing for some t _x,t_y and 
t _ resolution , the set 

20 _y,t _ resolutiOTiy t _ bitPlane)\ T<.t_ bitPlane < maxPlaneBit {t _ resolution) 

is requested where T is the mmimal bit plane required to the current view. Here, where 
the luminance mapping is taken in account, the value of T might be increased or 
decreased. 



25 The number of bit planes reduced (added) from the request list is 

liogj ! ^11 . 

L \Maxhnal _ gradient )\ 
Le., for those t^x^t^y and /^reyo/w/zon mentioned before, the foUowiug set is 
requested: 

{{i ^^^^ ^y>t ^resoIution,t _bitPlane)\T' _bitPlane^m€ixPl^ 



30 where 



•■=r.[.og.[- 



Jdaximal ^gradient ^ 

Examples: 
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5 1, Given 

• Image depth of 12-bit 
Screen depth of 8-bit 

• Linear himinance mapping. I.e., Maximal gradient = 2"^ • 
The nmnber of bit planes reduced from the request list is: 



10 



[Maximal gradient]\ ^ [^]\ ^ ^ 



2. Given a luminance mapping with Maximal gradient =2 

The number of bit planes reduced from the request list is: 



Thus one bit plane is added to the original set 
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6.3 Step 403: Encoding the request list 

The client imaging module in the client computer 110 encodes the request list 
into a request stream that is sent to the server computer 120 via the commimication 
20 network 130 (Figure 1). The request list encoding algorithm is a simple rectangle-based 
procedure. The heuristics of the algorithm is that the requested data block usually can 
be groiQsed into data block rectangles. From the request list of data blocks indexed by 
the encoding algorithm computes structures of the type 

25 {(t _x,t _y,t _resoltaion^t _bitPlane),n^,ny^ ^ ^x>^y ^1 

(1.3) 

Each such structure reprraents the x jiy data blocks 

{{t _x^i^t _y + _ resolution, t _ bitPlane)} , O^i <n^,0:^ J Kn^ 



30 
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5 The encoding algorithm attempts to create the shortest possible list of stractuxes, 

collecting the data blocks to the largest possible rectangles can do this. It is important 
to note that the algorithm ensures the order of data blocks in the request Ust is not 
changed, since the server 120 will respond to the request stream by transmitting data 
blocks in the order in which they were requested. A good example of when Hub works 
1 0 well is when a tiser zooms in into a ROI at a high resolution that was never viewed 

before. In such a case the request list might be conq)osed of hundreds of requested data 
blocks, but ttiey will be collected to one {x,y) rectangle for each pair 
{t resolution, t _ bitPlane) . 

15 

6.4 Step 404: Receiving the data blocks 

The client computer 1 10 upon receiving from the server computer 120 an 
encoded stream containing data blocks, decodes the stream and inserts the data blocks 
into their appropriate location in the distributed database using their ID as a key. The 

20 simple decoding algorithm perfomied here is a reversed step of the encoding in&a. 

Since the cUent 110 is aware of the order of the data blocks in the encoded stream, only 
the size of each data block need be r^orted along with the actual data. In case the 
server 120 informs of an CTipty data block, the receiving module marics the appropriate 
slot in the database as existing but empty. 

25 Recall that the subband tile associated with each data.block is denoted by the 

first three coordinates of the four coordinates of a data block (t _x,t _y,t ^resolution) . 

From the subband tile's coordinates the dimensions are calculated of the area of visual 
significance; that is, the portion of the ROI that is affected by the subband tile. Assume 
that each subband tile is of length tileLength and tbat the wavelet basis used has a 

30 maximal filter size maxFilterSize ^ then defining hFilterSize \7naxFilterSize/2] and 
factor := numberOfResolvtions — t _ resolution + 1 , we have that the dimensions of the 
affected region of the ROI (in the original image's coordinate system) are 
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5 



X + 1) X tileLength^'^'*' + hFilterSize^'^'' ] 



These dimensions are merged into the next rendering operation's region. The 
rendering region is used to efGciently render only the updated portion of the ROL 

10 

6.5 Progressive Rendering 

During the transmission of ROI data firom the server to the client, the client 
performs rendering operations of the ROI. To ensure that these rendering tasks do not 
intexropt the transfer, the client runs two program threads: commimications and 
15 rendCTng. The rendering thread runs in the backgroxmd and draws into a pre-allocated 
"oflf-screen" buffer. Only then does the client use device and system dq>endant tools to 
output the visual information &om the '^off-screen*' to the rendering device such as the 
screen or printer. 



progressive rendering is performed only for the portion of the ROI that is affected by 
newly arrived data. Specifically, data that arrived after the previous rendering task 
began. This **updated region" is obtained using the method of step 404 described in 



The rendering algorithm performs reconstruction of the ROI at the highest 
20 possible quality based on the available data at the client. That is, data that was 
previously cached or data that "just" arrived fix)m the server. For efficiency, the 



25 §6.4. 



The parameters of the rendering algorithm are composed of two sets: 



30 



li! The ROI parameters described in Table 3. 

2. The parameters transmitted from the server explained in Table 5 , with the 
exception of the /MwpS'ize parameter, which is a "searver only" parameter. 



The rendering algorithm computes pixels at the dyadic resolution 
dyadicRresolution(ROI}. Recall that this is the lowest possible dyadic resolution that is 
higher than the resolution of the ROL The obtained image is then resized to the correct 
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5 resolution. Using a tiling of the multiresolution represmtation of the ROI, the steps of 
tiie algorithm are perfonned on a tile by tile basis as described in Figure 1 1 . Since the 
tiles' lengtti are tileLength, which is typically chosen as 64, the rendering algorithm is 
memory efiBd^t. 

10 6.S.1 The rendering rate 

As ROI data is transmitted to the client 1 10, the rendering algorithm is 
performed at certain time intervals of a few seconds. At each point in time, only one 
rendering task is performed for any given displayed image. To ensure that progressive 
rendering does not become a botfleneck, two rates are measured: the d4ta block transfer 

15 rate and the ROI rendering speed. If it predicted that the transfer will be finished 

before a rendering task, a small delay is inserted, such that rendering will be perfomied 
after all the data arrives. Therefore, in sl slow network scenario (as the Internet often 
is), for ahnost the entire progressive rendering tasks, no delay is inserted. With the 
arrival of every few kilobytes of data, containing the information of a few data blocks, 

20 a rendering task visualizes the ROI at the best possible quality. In such a case the user 
is aware that the bottleneck of the ROI rendering is the slow network and has the option 
to accept the current rendering as a good enough approximation of the image and not 
wait for all the data to arrive. 



25 6.S.2 Memory constraint subband data stmcture 



This data-stractm« is required to efBciently store subband coef&cients, in 
memory, during the rendering algoritbm. This is required since the coefBcients are 
represented in long integer (lossless coding mode) or floating-point Qossy coding 
30 mode) precision which typically require more memory than pixel representation (1 
byte). In lossy mode, the coefficients at the client side 1 10 are represented using 
floating-point representation, evCTi if they were confuted at the server side 120 using 
an integer implementation. This will minimize roimd-ofT errors. 

At the beginning of the rendering algorithm, coefficient and pixel memory strips 
35 are initialized, dyadicWidth{ROl) msy be denoted as the width of the projection of the 
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5 ROI onto the resolution dyadicResolution{^ROI^ . For each component and resolution 
1 < 7 < dyadic^G solution {ROT) , four subband strips are allocated for the four types of 
subband coefficients: hl,lh,hh and Ha(fBiL The coefficient strips are allocated with 



10 



15 



dimensions 



For each component and resolution l^j< dyadicResolution a pixel strip is 
allocated with dimensions 



Beginning with the lowest resolution 1, the algorithm proceeds with a recursive 
multiresolution march from the top of the ROI to bottom (y direction. Referring to 
Figures 10 and 1 1, in step 11 01, the multiresolution strips are filled with sub-tiles of 
coefficients 1050 decoded from the database or read from the memory cache. Fxom the 

20 coefficients we obtain multiresolution pixels 1051 using an inverse subband transform 
step 1 102 (shown in further detail in Figure 10). Each time a tile of pixels at 
resolutions j < dyadicResoluHon{ROl) is reconstructed, it is written into the pixel strip 
at the resolution j . Each time a tile of pixels at the highest resolution 
dyadicResolution (^ROl) is reconstructed, it is fed into the inverse color transform and 

25 resizing steps 1 103, 1 104. 



6*53 Step 1101: decoding and memory caching 

The subband coefficients data structure described previously in section 6.5.2 is 
30 filled on a tile basis. Each such subband tile is obtained by decoding the corresponding 
data blocks stored in the database or reading fiom the memory cache. The memory 
cache is used to store coefficients in a simple encoded format The motivation is this: 
the decoding algorithm described previously in section 5.2 is computationally intensive 
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5 and thus should be avoided whenever possible. To this end the rendering module uses 
a tnemoiy cache 111 where subband coefQcient are stored in very simple mcoded 
format which decodes very fast. For each required subband tile» the following 
extraction procedure is performed, described in Figure 25, beginnmg at step2S01 . Jn 
step2502, if no data blocks are available in the database for the subband tile, its 

10 coefficients are set to zero (step2503). In step2504, if the tile's memory cache storage 
is updated, namely it stores coefficients in the same precision as in the database, then 
the coefficients can be efficiently read fiom there (step2505). In step2S06, the last 
possibility is fliat tihe database holds the subband tile in hi^^er precision. Then, the tile 
is decoded down to the lowest available bit plane using the algorithm previously 

15 described in section 5.2 and the cached representation is replaced with a representation 
of this higher precision information. * 

6.5.4 Step 1102: inverse lossless wavelet transform 

This is an inverse step to step 603 performed in the server (see 7.1.5). Following Figure 
20 21 we see that four **extended" subband coefficient sub-tiles of length 

tileLength/2 + maxFilterSize at the resolution / are read from the coefficient strips data 
structure and transfomied to a tile of pixels at the next higher resolution using 
losslessWaveletdTransformType{i) . If i +1 < dyadicResolution (ROI), the tile of pixels 
obtained by this step is inserted into the pixel memory strip at the resolution /+ 1 . If 
25 J + 1 = dyadicResolution {ROl) the tile of pixels is processed by the next step of color 
transform. 

Remark: Recall from 5.1 •! that the "half bits" are initialized as zeros, dierefore the 
inverse step is well defined even if their **real" value is not available in the climt yet. 

30 6.5.5 Step 1103: inverse color transform 

This is an inverse step to step 603 perfomied at the server 120. It is performed 
only for tiles of pixels at the resolution highestSubbandResohUion - At this stage, all of 
the pixels of each such tile are in the ontputColorSpace and so need to be transformed 
into a displaying or printing color space. For example, if the original image at the 
35 server 120 is a color image in the color space RGB, the pixels obtained by the previous 



wo 01780561 FCTAJSOJ/12390 

45 

5 step of inverse subband transform are in the compression color space YUV, To convert 
back from YUV to RGB, we use the inverse step described in Figure 13. If the ROI is 
at an exact dyadic resolution. Him the tile of pixels in the rendering color space is 
written into the ofif-screen buffer. Else it is resized in the next step. 



10 

6.5*6 Step 1104: image resize 

In case the resolution of the ROI is not an exact dyadic resolution, the image 
obtained by the previous step must be re-sized to this re^lution. This can be 
accomplished using operating system imaging functionality. In most cases the 

IS operating system's implementation is sub-sampling which produces in many cases an 
aliasing effect which is visually not pleasing. To provide higher visual quality, the 
imaging system of the present invention may use the method of linear intetpolation, for 
example described in J. Proakis and D. Manolakis, 'TMgital signal processing". Prentice 
Hall, 1996. The output of the inteipolation is written to the off-sere^ buffer. From 

20 there it is displayed on the screen using system device dependant methods. 

6SJ7 Step 1105: mapping to 8-bit screen 

When luminanceMap is active mapping to 8-bit screen is performed using the 
mapping function described in 6.1.1. 

25 

7. SERVER WORFLrOW 

With reference to Figure 5, the operation of the server computer 120 (Figure 1) 
will now be described. Initially, an uncompressed digital image is stored in, for 
example, storage 122 of the server computer 120. This uncompressed digital image 
30 may be a two-dimensional image, stored with a selected resolution and depth. For 
example, in the medical field, die imcompressed digital image may be contained in a 
DICOM ffle. 

Once the client computer 1 10 requests to view or print a certain image, the 
server performs the preprocessing step SOL This step is a computation done on data 
35 read from the original digital image. The results of the computation are stored in the 
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5 server cache device 121. After this fast computation a **ready to serve" message is sent 
from the server to the cHent containing basic infomiation on the image. 

In step 502, the server receives an encoded stream of requests for data blocks 
associated with a ROI that needs to be rendered at the cUent. The server then decodes 
the request stream and extracts the request list 
10 In step 503, the server reads from cache or encodes data block associated with 

low resolution portions of the ROI, using the cached result of the preproc^sing stage 
501, 

If the ROI is a high-resolution portion of the image, the server, in step 504, 
reads from cache or performs a "local" and ef5cient version of the preprocessing step 
15 501. Specifically^ a local portion of the unconq>ressed image, associated with the ROI, 
is read from the storage 122, processed and encoded. In st^ 505, the data that was 
encoded in steps 503-504 is progressively sent to the cUent in the order it was 
requested. 



20 7.1 Step 501: preprocessing 

The preprocessing step is now described with respect to Figure 6. The 
preprocessing algorithm's goal is to provide the fastest response to the iiser's request to 
interact with the image. Once this fast computational step is performed, the server is 
able to provide efficient **pixel-on-demand" transmission of any client ROI requests 

25 that will follow. In most cases the first ROI is a view of the full image at the highest 
resolution that "fits" the viewing device. The preprocessing algorithm begins with a 
request for an uncompressed image that has not been processed before or has been 
processed but the result of this previous computation has been deleted fix>m the cache. 
As explained, this unique algorithm replaces the possibly simpler procedure of 

30 encoding the fiill image into some progressive format. This latter technique will 
provide a much slower response to the user's initial request then the technique 
described below. At the end of the algorithm a **ready to serve ROI of the image" 
message is sent to the client containing basic information on the image. While some of 
this ioformation, image dimensions, original color space, resolution etc., is available to 

35 the user of the ctient computer, most of this information is 'internal" and required by 
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5 the cheat to fonntilate ROI request Usts (§6.2) and progressively render (§6.5). Next we 
describe in detail the preprocessing algorithm. 

7.1*1 Preprocessing Parameters 
Table? 

10 



Variable 


Iv4[eaning 


losslessMode 


If true, preprocessing will prepare data that can be used for 
lossless transmission. 


SUOOanuI. rCUVSjOTlHiypi 


' 1 'Hp ■firpi'm p'wriTlr iilloAX/c tint* siViiHfx/ fr\ ci=»lf»r»i' a rW'f-Ff^-r^n^f ctiVyV^a-nA 

XlX^ Xi.<uJlxw VV^JLJ\. cLLxyjyVo illw <UJ111.LY iv^ oC?lCVv Ct> vXliJLwirCXit otlOO<4Xlil 

transform for each resolution of each image. The technical term 
is: non-stationary transforms. 


numberOfResolutions 


The number of resolutions in the Multiresolution stmcture 
calculated for the image. 

Typically, numberOfResolutions = log^ ^-yjlmageSize^ . 


JumpSize 


A number in the range [O, numberOfResolutions — l] . The 

preprocessing stage computes only the top lower part of the 
image's multiresolution pyramid of the 


tileLength 


Typically = 64. Tradeoff between time and coding performance. 


nTilesX{j) 


Number of subband tiles in the x direction at tiie resolution J 


nTilesX{j) 


Nimiber of subband tiles in the y direction at the resolution j 


inputColorSpace 


Color space of uncompressed original image. 


outputColorSpace 


Transmitted color space used as part of the encoding technique. 


numberOfComponents 


Number of components in OutputColorSpace . 


threshold (c^j) 


A matrix of values used in lossy compression. The subband 
coefficients of the component c at the resolution j with absolute 

value below ^AreyAo/rf(c,7) are considered as (visually) . 

insignificant and set to zero. 



Given an ii^ut image, the parameters described in Table 7 are computed or 
chosen. These parameters are also written into a header sent to the client 110 and are 
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5 used during tiie progressive rend^ing step 405 (see section 6.5, described previously). 
The important parameters to select are: 

1 . losslessMode : In this mode, progressive transmission of images takes 
place until lossless quality is obtained Choosing this mode requires the 

10 preprocessing algorithm to use certain reversible wavelet transforms, 

and can slow down tiie algorithm. 

2. subbandTransformType(^j) : The (dynamic) selection of wavelet basis 

(as described, for example, in L Daubechies, *Ten lectures on wavelets**, 
1 5 Siam, 1992) is crucial to the performance of the imaging system. The 

selection can be non-stationaiy, meaning a dLBferent transform for each 
resolution J . The selection is derived firom the following 
considerations: 

a. Coding performance (in a rate/distortion sense): Iliisis 

20 obviously required from any decent subband/wavelet transform. 

b. Approximation of ideal low pass: It is &vorable to choose a 
transform such tiiat low resolutions of the image will be of high 
visual quality (some filters produce poor quality low resolutions 

. even before any compression takes place). 

25 c. Fast transform implementation: Can the associated fast transform 

be implemented using lifting steps (as described, for example, by 
I. Daubechies and W. Sweldens, 'TFactoring wavelet transforms 
into lifting stq)s", J. Fourier Anal. Appl., Vol. 4, No. 3, pp. 247-" 
269, 1998), using only integer shifts and additions, etc. Some 

30 good examples are the Haar and CDF transforms (1 3), (2,2)*** 

described in I. Daubechies, '*Ten lectures on wavelets*', Siam, 
1992. 

d. Fast low pass implementation: A very important parameter, 
since together with the parameter jumpSize ^ it dctGrniines 
35 almost all of the complexity of the algorithm. For example, the 
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CDF (1,3) is in this respect the "optimal" transform with three 
vanishing moments. Since the dual scaling function is the simple 
B-spline of order 1, its low pass is simple averaging. Thus, the 
sequence of CDF transforms, using the B-spline of order 1 as the 
dual scaling function, but with wavelets with increasing number 
of vanishing moments are in some sense optimal in the present 
system. They provide a framework for both real time response 
and good coding efficiency. 

Lossless mode: IflosslessMode is tme we must choose the 
filters from a subclass of reversible transforms (see, for example, 
**Wavelet transforms that map integers to integers**, A. 
Calderbank, 1. Daubechies, W. Sweldens, B. L* Yeo, J. Fourier 
Anal. AppL, 1998). 

Low system I/O: If the network 130 in Figure 1 connecting 
between the Image residing on the storage 122 and the imaging 
server 120 is slow, the bottleneck of the preprocessing stage (and 
the whole imaging system for that frict) might be simply the 
reading of the original image. In such a case a transform may be 
chosen with a lazy sub-sampling low pass filter that corresponds 
to efficient selective reading of the input image. Many 
interpolating subband transforms with increasing number of 
vanishing moments can be selected to suit this requirement. 
However, this choice should be avoided whenever possible, since 
it conflicts with (a) and (b). 

Image type: If the type of the image is known in advance, an 
appropriate transform can be chosen to increase coding 
efficiency. For example: Haar wavelet for graphic images, 
smoother wavelet for real-life images, etc. In the graphic arts 
field, there are numerous cases of documents composed of low 
resolution real-life images and high resolution graphic content. 
In such a case, a non-stationary sequence of transforms may be 
chosen: Haar for the high resolutions and a smoother basis 
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5 starting at the Mghestresolxition of a real-life images In case 

of low system I/O (f), a non-stationaiy choice of inteipolating 
transfoims of dijOTerent orders is required. 

3. jumpSize\ This parameter controls the tradeoff between fast response 
10 to the user's initial request to mteract wifli the image and response times 

to subsequent ROI requests. When JnmpSize is large, the initial 
response is faster, but each computation of a region of interest witti 
higher resolution than the jump might require processing of a large 
portion of the original image. 
1^ 4. InputColorSpace: The iapai color spaces siq>ported in lossless mode are: 

a. Qrayscale: For grayscale images 

b. RGB 

5. otd^ptitColorSpacei The following are color spaces which perform well 
20 in coding: 

a. Grayscale: For grayscale images 

b. YUV: for viewing color images 

Referring to Table 7 [LP], losslessMode is set to true. Threshold (c, j) is not in 

25 use, since in lossless mode, there is no thresholding. The rest of flie variables have the 
same meaning as in the lossy algorithm. 



7.1.2 Memory constraint multiresolution scan data stractare 

30 Most wavelet coding algorithms have not addressed the problem of memory 

complexity. Usually the authors have assumed there is sufficient memory such that the * 
image can be transformed in memory fiom the time domain to a wavelet frequency 
domain r^resentation. It seems tiie upcoming JPEG2000 will address this issue, as did 
its predecessor JPEG. The preprocessing algorithm also requires performing subband 

35 transforms on large images, although not always on ttie full image, and thus requires 
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' 5 carefijl memory management This means that the memoiy usage of the algorithm is' 
not of the order of magnitude of the original image, as described in J, M. Sh^iro,**An 
embedded hierarchical image coder using zero-trees of wavelet coef5cients*% lEBE 
Trans. Sig. Proc, Vol. 41, No. 12, pp. 3445-3462, 1993. 

GivOT an uncompressed image we allocate the following number of memory 

10 strips 

ntanberOfComponents x (nwnberOfResolutiorts — jumpSize) 



15 



of sizes 



[2-^""^^*^'^^> image Width, 3 x tileLength 1 2 -h maxFilterSiz^ 
for 1^ J <^ numberO/ResoltUions — jumpSize— X and 
20 [image Width, tileLength + 2 x maxFilterSize] 

for J = numberOfResolutions — JumpSize 



That is, the memory usage is proportional to 2"^"'^' x imageWidth . Each such 
25 strip stores low-pass coefBcients in the color space outputColorSpace at various 
resolutions. 

Referring to Figure 6, during the preprocessing stage, the resolutions are 
scanned simultaneously from start to end in the y direction. For each color component 
and resolution, the corresponding strip stores low-pass coefScients or pixels at that 
30 resolution. The core ofthe preprocessing algorithm are steps 604-607, where tUes of 
pixels of length tileLength + 2x maxFilterSize are read from the memory strips and 
handled one at a time. In step 604 the tile is transformed into a tile of length tileLength 
containing two types of coefficient data: subband coefficients and pixels at a lower 
resolution. The subband coefficients are processed in steps 605-606 and are stored in 
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5 the cacbe. The lower resohxtion pixels are inserted in step 607 to a lower resolution 
memory strip. Whenever such.a new sub-tile of lower resolution pixels is inserted into 
a strip, the algorithm's memory management module performs the following check: If 
the part of the sub-tile exceeds the current virtual boundaries of the strip, the 
corresponding first lines of the strip are considered unnecessaiy anymore. Their 
1 0 memory is (virtually) re-allocated for the puipose of storing the sub-tile' s new data. 



7.13 Step 601: Lossless color-transform 

This step is xises the conversion formula described in Figure 13. This step must 
be perfomied before step 602, because the lossless color conversion is non-linear. 

15 

7.1.4 Step 602: Lossless Wavelet Low Pass 

The motivation for the low pass step is explained in § 6.1.4 in the above-cited 
Ser. No. 09/386,264, which disclosure is incorporated herein by reference. In a lossless 
mode there are a few emphasis that are represented here. 
20 In step 602, the low pass filters of the 

transfomis subbandTransformType ( 7 ) , 

numberOfResolutions - jumpSize numberOfResolutions , are used to obtain a low 
resolution strip at the resolution ntanberCSfResohaions JtmipSize (as can be seen in 
Figure 26). Typically, it is required to low pass about 1^""^^ lines of the original 

25 image 1010 to produce one line of the low resolution image. The low pass calculation 
is initiated by a read of tiles of pixels firom the memory strqps performed in step 604. 
Whenever there is an attempt to read missing low resolution lines, they are computed 
by low passing the original image and inserted into the memory strip. The insertion 
over-writes hues are no long^ required, such that the algorithm is memory constiained. 

30 In the case where a non-linear color transform took place in the previous step 601, the 
results of that transform are low-passed. 

In the lossless mode of operation, the jumpSize parameter defines the number of 
lossless wavelet low pass steps should be done. A single low pass step is the same for 
Haar and CDF (1,3) and defined by the following two stages (taken fi-om (3.20) and 

35 (3.22): 
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x{2r 



X-direction: s(n) = 



2 J 



Y-direction: s(n) = x(2n)-{'x(2n + l)\ 
Namely, in a 2D representation, the low pass step is defined by 

10 For JumpSize = 1 and jumpSize = 2 (other sizes practically are not needed), the server 
performs these steps efficiently (almost like the lossy algorithm) by a single operation 
that simulates exactly JumpSize low pass steps defined in (7.1). As noticed from (7.1), 
the simplicity of the formula makes filters such as Haar and CDF (1,3) ^'optimal" in the 
respect of low pass efficiency. 

15 

7.1.5 Step 603: Forward lossless Wavelet transform 

In Step 603 we perform one step of an efficient local lossless wavelet transform 
(§3), on a tile of pixels at the resolution l^j^ ntanberOfResolutions — jumpSize - The 

type of transform is determined by the parameter losslessWav€JetTranrformType(^j) . 

20 As described in Figure 1, the transform is performed on an "extended^' tile of pixels is 
of length tileLength + 2 x maxFilterSize (unless we are at the boundaries), read directly 
firom a multi-r^olution strip at the resolution y +1 . The output of the step is a lossless 
subband tile composed of wavelet coefficients including Half bit coefficients and low 
resolution coefficients/pixels. The transform step is efficiently implemented in integers 

25 as described in §3. 

The subband transform of step 603 outputs three types of data: scaling function 
(low pass), wavelet (high pass) and Halfbits. The wavelet coefficients are treated in 
step 604 while the scaling function coefficients are treated in step 605. 

Remark: Tiles of pixels which are located on the boundaries sometimes need to 

30 be padded by extra rows and/or columns of pixels, such that they will formulate a **fidr' 
tile of length tileLength. 



I 
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5 7.1.6 Step 604: Variable Length encoding and storage 

In step 604, flie subband coefficieaits that are calculated in step 603 are variable 
lengfli encoded and stored in the cache 121. If maxBitPlane{tile) = 0 we do not write 
any data. Else we loop on the coefficient groups {coef (2x z + ;r, 2x y + y)] . For 
each such groiqj we jBrst write the group's variable length length{ij) using 

0 logj {maxBitPlane(tile)) bits. Then for each coefficient in the group we write 
length (i , 7) +1 bits representing the coefficient's value. The least significant bit 
represents the coefficient's sign: if it is 1 then the variable length encoded coefficient is 
assumed to be negative. The HalfBit subband coefficients are written in one-bit per 
coefficient. 



7.1.7 Step 605: Copying low pass coefficients into the moltiresolntion strip 
stmctore 

In step 503, unless losslessModeis true, the subband coefficients calculated in 
step 604 are quantized. This procedure is performed at this tune for the following 
reason: It is required that the coefficients computed in the previous step will be stored 
in the cache 121. To avoid writing huge amounts of data to the cache, some 
compression is required. Thus, the quantization step serves as a preparation step for the 
next variable length encoding step. It is important to point out that the quantization 
step has no effect on compression results. Namely, the quantization step is 
synchronized with the encoding algorithm such that the results of the ^coding 
algorithm of quantized and non-quantized coefficients are identical. 

A tile of an image component c at the resolution J is quantized using the given 
threshold threshold (c,y) : for each coefficients x , the quantized value is 
[x/ threshold (c, y ) J , It is advantageous to choose the parameters threshold (c, j) to be 

dyadic such that the quantization can be implemented using integer shifts. The 
quantization procedure performed on a subband tile is as follows: 
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1. Initialize TnaxBitPlane{iile)=Q. 

2. Loop over each group of four coefficients 
{coef (2xz + jc,2x 7 + j')}, j^,- each such group initialize a variable 

length parameter length (/, y ) = 0 . 

3. Quantize each coefficient in the group coef (2 x i + x, 2 x / + j;) using the 
^propriate threshold. 

4. For each coeflBcient, update length {iyj) by the bit plane b of the 
15 coefBcient, where the bit plane is defined by 

\coef (2 X / + jc, 2 X y + e [2* threshold (c, j) , 2^^ threshold {c^ j)) 

5. After processing the group of four coefficients, use the final value of 
20 fe/i^A(i,j) to update maxBitPlane{tile) by 

maxBitPlane {tile) = max(maxBitPlane (tile) , length (f , y )) 

6. At the end of the quantization step, store the value maxBitPldne(tile) in 
25 the cache 121. 

Note that for subband tiles located at the boundaries we can set to zero subband 
coefi&cients that are not associated with the actual image, but only with a padded 
portion. To do this we take into account the amount of padding and the parameter 
30 maxFilterSize . The motivation for the **removal" of these coefBicients is coding 
ef5ciency. 
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S 7.2 Step 502: Decoding the request stream 

This is the inverse stesp of section 6.3. Once the request stream arrives at the 
server 120, it is decoded back to a data block request list Each data structure the type 
r^resenting a group of requested data blocks is converted to the sub-list of these data 



10 



blocks. 



73 Step 503: Encoding low resolution part of ROI 

Step 503 is desoribed in Figure 7. It is only performed whenever the data 
blocks associated with low-resolution subband tile are not available in the server cache 
121. 

15 Step 701 is the inverse step of step 604 described in §7,1.6. Jn the preprocessrag 

algorithm subband tiles of lower resolution, that is resolutions lower than 
numberOfResolutions - jumpSize , were stored in the cache using a variable length type 
algorithm. For such a tile we first need to decode the variable length representation. 
The algorithm uses the stored value maxBitPlane{tile) . 

20 1 . If fnaxBitPlane{tile) = 0, flien all the coefificiente are set to zero including the 

HalfBit subband. 

2. If maxBitPlane{tile) = 1 , then all the ooefiScients are set to zero, and the HalfBit 
subband coefficient are read bit by bit firom cache. 

3. Else, as performed in 2, the HalfBit subband coefficient are read bit by bit from 
25 cache, and we perform the following simple decoding algorithm: 

For each group of four coefficients \coef{2x,i^x,2^ J^y^x we read 
Iog2 {maxBitPlane(tile)) bits representing the variable length of the group. 

Assume the variable length is length(i,j) , For each of the four coefficients we 
30 then read length (?, j) -h 1 bits. The least significant bit represeats the sign. The 
reconstracted coefGcient takes the value: 
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-1 readBits Stl^l 
readBits &1-0 



In step 702 we use the encoding algorithm desoibed in §5.1 to encode the 
requested data blocks associated with the extracted subband tile. 

10 7.4 Step 504: Processing high resolution part of ROI 

Stqp 504 is described in Figure 8. In case we have ixsed JumpSize > 0 in step 
501 and the resolution of the ROI > numberOfResolutions -JumpSize, we are sometimes 
required to perform a local variation of the preprocessing step described in §7. 1 . 
Whenever the server receives a request list of data blocks we check the following. If a 

15 data block has been previously computed (present in the cache 121) or is associated 
wi& a low resolution subband tile data block then it is either sini^ly read from the 
cache or handled in step 503. Else, the coordinates of the data block are used to find the 
'^minimal" portion of the ROI that needs to be processed. Then, a local version of the 
preprocessiag algorithm is performed for this local portion. The difference here is that 

20 step 804 replaces Variable Length coding stqp 604 of the preprocessing algorithm by 
the encoding algorithm given in §5.1. 

7.5 Step 505: Progressive transmission of ROI 

In the final step, the encoded data tiles are sent firom the server 120 to the client 
25 1 10, in the order they were requested. la many cases, data blocks will be empty. For 
example, for a region of the original image with a constant pixel value, all of the 
corresponding data blocks will be empty, except for ttie first one that will contaia only 
one byte with the value zero representing maxBitPlane(tile) = 0 . For a low activity 

region of the image, only the last data blocks representing higher accuracy will contaui 
30 any data. Therefore, to avoid the extra side information, rectangles of empty data 

blocks are collected and reported to the client 110 under the restriction that they are 
reported in the order in which they were requested. For blocks containing actual data, 
only the data block's size in bytes need be reported, since the client 110 already knows 
which data blocks to expect and in which order. 
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5 Tbe present iavention has been described in only a few embodiments, and with 

respect to only a few applications (e.g., commercial printing and medical imaging). 
Those of ordinary skill in the art will recognize «iat the teachings of the present 
invention may be used in a variety of other ^plications where images are to be 
transmitted over a commmucation media 
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What is claimed: 

1 . A system for transmitting a digital image over a communication network, 
conqprising: 

(a) an image storage device for storing a digital image; 

(b) a client computer coupled to the commmiication network, wherein the client 
computer generates and transmits across the communication network 
coordinates defining a region of interest within the digital image; 

(c) a server computer, coupled to the communication netwoik and the image 
storage device, wherein the server computer performs the steps of: 

(i) pre-processing the digital image through a lossless wavelet 
transformation; 

(ii) receiving the coordinates jQrom the client computer; and 

(iii) progressively transmitting to the client computer the region of 
interest within the digital image defined by the coordinates. 

2. The system of claim 1, wherein the server computer progressively transmits the 
region of interest to a selected quality threshold. 

3. The system of claim 1, wherein the server computer progressively transmits the 
region of interest to lossless quality. 

4. The system of claim 1, wherein the client computer reverse transforms the region of 
interest received firom the server computer, to form a lossless reproduction of the digital 
image. 

5. The system of claim 4, wherein client computer displays the lossless reproduction of 
the digital image on a web browser residCTit on the client computer. 
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6, The system of claim 1, wherein the server computer performs the pre-processing step 
through a lossless wavelet transformation conq>rising two non-identical one- 
dimensional transforms. 

7. A system for transmitting a digital image over a communication network, 
conq>rising: 

(a) an image storage device for storing a digital image; 

(b) a client computer coupled to the communication network, wherein the 
client computer generates and transmits across the communication 
network coordinates de£nmg a region of interest within the digital 
image; 

(c) a server computer, coupled to the conamunication network and the image 
storage device, wherein the server computer performs the steps of: 

(i) pre-processing the digital image through a lossless wavelet 
transformation; 

(ii) generating wavelet coefficients corresponding to the digital 
image; 

(iii) generating half-bit flags, each half-bit flag corresponding to 
a wavelet coefficient 

(iv) receiving the coordinates froin the client computer, 

(v) progressively transmitting to the client computer the region 
of interest within the digital image defined by the coordinates, 
based on the wavelet coefficients and the corresponding half-bit 



8. The system of claim 7, wherein the server computer progressively transmits the 
region of interest to a selected quality tlu^hold. 



9. The system of claim 7, wherein the server computer progressively transmits the 
region of interest to lossless quality. 
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10. The system of claim 7, wherein the client computer reverse transforms the region of 
interest received from the server computer, to form a lossless reproduction of the digital 
image. 

11. The system of claim 10, wherein client computer displays the lossless reproductioh 
of the digital image on a web browser residrat on the client computer. 

12. The system of claim 7, wherein the server computer performs the pre-processing 
step through a lossless wavelet transformation comprising two non-identical one- 
dimensional transforms. 

13- A method for transmittmg a digital image from a server computer to a client 
computer, wherein the client computer generates coordinates defining a region of 
interest within the digital image, and transmits the coordinates to the server computer, 
the method comprising the steps of: 

(a) storing a digital image within an image storage device; 

(b) pre-processing the digital image through a lossless wavelet 
transformation; 

(c) receiving the coordiaates at the server computer from the client 
computer; and 

(d) progressively transmitting from the server computer to the client 
computer the region of interest within the. digital image defined by the, 
coordinates. 

14. The method of claim 13, wherein the server computer progressively transmits the 
region of interest to a selected quaUty threshold. 
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15. The method of claim 13, wherein the server computer progressively transmits the 
region of interest to lossless quality. 

16. The method of claim 13, farther comprising fee step of reverse transfbiming at the 
client computer the region of inter^t received fit>m the server computer, to form a 
lossless reproduction of the digital image. 

17. The method of claim 16, further comprising the step of displaying at the client 
computer the lossless reproduction of the digital image on a web browser resident on 
the client computer. 



18. The method of claim 13, wherein the pre-processing step is performed through a 
lossless wavelet transformation comprising two non-identical one-dimensional 
transforms. 



19. A method for transmitting a digital image from a server computer to a client 
computer, wherein the client computer generates coordinates defining a region of 
interest within the digital image, and transmits the coordinates to the s^ver computer, 
the method comprising the steps of: 

(a) storing a digital image within an image storage device; 

(b) pre-processing the digital image through a lossless wavelet 
transformation; 



(c) receiving the coordinates at the server coniputer from the client 
compntsr, and 

(d) generating wavelet coef&cients corresponding to the digital image; 

(e) generating half-bit flags, each half-bit flag corresponding to a 
wavelet coefiBcient; 
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(f) receiving the coordinates at the server computer firozn the client 
compute; and 

(g) progressively transmitting from the server computer to the client 
computer the region of interest Avithin the digital image defined by the 
coordinates, based on the wavelet coefficients and the corresponding 
half-bit flags. 

20. The method of claim 19, wherein the server computer progressively transmits the 
region of interest to a selected quality threshold. 

21 . The method of claim 19, wherein the server computer progressively transmits the 
region of interest to lossless quality. 

22. The method of claim 19, further comprising the step of reverse transforming at the 
client computer the region of interest received from the server computer, to form a 
lossless reproduction of the digital image. 

23. The method of claim 22, further comprising the step of displaying at the client 
computer the lossless reproduction of the digital image on a web browser r^ident on 
the client computer. 

24. The method of claim 19, wherein the pre-processing st^ is performed through a 
lossless wavelet transformation comprising two non-identical one-dimensional 
transforms. 

25. A server computer for transmitting a digital image to a client computer, wherein the 
client computer generates and transmits to the server computer coordinates defining a 
region of interest within the digital image, the server computer comprising: 

(a) an image storage device for storing the digital image; 
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(b) a im>cessor for perfonning the steps of: 

(i) pre-processing ttie digital image flirough a lossless wavelet 
ttansfoxmatioxi; 

(ii) receiving the coordinates fix>m the client computer; and 

(iii) progressively transmitting to the client computer the region of 
interest within the digital imag6 defined by the coordinates. 

26. The system of claim 25, wherein the server computer progressively transmits the 
region of interest to a selected quahty threshold. 

27. The system of claim 25, wherein flie server computer piogtessively transmits the 
region of interest to lossless quality. 

28. The system of claim 25, wherein the client computer reverse transfomis the region 
of interest received from the server compiiter, to form a lossless reproduction of the 
digital image. 

29. The system of claim 28, wherein client computer displays the lossless reproduction 
of the digital image on a web browser resident on the client computer. 

30. The system of claim 25, wherein the servCT computer performs the pre-processing 
step through a lossless wavelet transformation comprising two non-identical one- 
dimensional transfbrms.- 

31. A server comqputer for transmitting a digital image to a client conq>uter, wherein the 
cliei3t computer generates and transmits to the server computer coordinates defining a 
region of interest within the digital image, flie SCTver computer comprising: 

(a) an image storage device for storing the digital image; 
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(b) a processor for perfoiming fhe steps of: 

(i) pre-processing the digital image through a lossless wavelet 
transfonnatioii; 

(ii) generating wavelet coefficients corresponding to the digital 
image; 

(iii) generating half-bit flags, each half-bit flag corresponding to 
a wavelet coefficient; 

(iv) receiving the coordinates from fhe client conq>uter; 

(v) progressively transmitting to fhe client computer fhe region 
of interest within ibe digital image defined by the coordinates, 
based on the wavelet coefficients and fhe corresponding half-bit 
flags. 

32. The system of claim 31, wherein the server computer progressively transmits the 
region of interest to a selected quality threshold* 

33. The system of claim 31, wherein the server computer progressively transmits the 
region of interest to lossless quality. 

34. The system of claim 31, wherein the client computer reverse transforms tiie region 
of interest received from fhe server computer, to form a lossless reproduction of the 
digital image. 

35. The system of claim 34, wherein client computer displays the lossless reproduction 
of the digital image on a web browser resident on the client computer. 

36. The system of claim 31, wherein fhe server conqputer performs fhe pre-processing 
step through a lossless wavelet transformation comprising two non-identical one- 
dimensional transforms. 
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bitModeKstartModel () ; 

coefSignModeLstartModel Q ; 

while (encoder.moreCoef Q) { 
if (encoderisCoefReported ()) \ 

arithmet{c_encode_symbol (bitModel.encoder.reportedCoefPrec 
isionSt 0) ; 

} 

else { 

if (encoderisCoefExactZero ()) ; 
anthmefic_encode_symbol (zeroCoefModel^true) ; 
else { 

arithmetic_encode_syfnbol (zeroCoefModel.false) ; 
arithmetic__encode_symbol (coefSignModel.encoder.getCoe 
fSign 0) ; 
} 

\ 

\ 

FIG. 15a 



bitModeLstartModel () ; 

for (int I = 0 ; i < hBlockSize ; { 

for (int j = 0 ; j < hBlockSize ; j++) { 
orithmeticencode^symboi (bitModel, 
coefHolfBit [i] 0]) ; 
] 

\ 
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bitModel .startModeiQ ; 
zeroCoefModel.startModet() ; 
coefSignModeLstartModelQ ; 

decoder.initializeLSBPIaneCoefScan () ; 

while (decodermoreCoef (}) { 
if (decoder.isCoefReported ()) { 
If (decoderJsLHCoef ()) { 
decoder. updoteLSB (0) ; 

! 

else I 

decoder.updateLSB (arithmetic_decoder_symbol (bitModei)) ; 

! 

{ 

else \ 

if (idecoderJsLHCoef ()) \ 
if (larithmetic^decoder_^mboi (zeroCoefModet)) 

decoder.setLSB (arithmetic_decoder_symbol (coefSignMode 

D) ; 
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for (int i « 0 ; I < hBlockSze ; { 
for (int j = hBlockSize ; j ; j — ,p++) { 
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